We propose a novel method for temporally pooling frames in a video for thetask of human action recognition. The method is motivated by the observationthat there are only a small number of frames which, together, containsufficient information to discriminate an action class present in a video, fromthe rest. The proposed method learns to pool such discriminative andinformative frames, while discarding a majority of the non-informative framesin a single temporal scan of the video. Our algorithm does so by continuouslypredicting the discriminative importance of each video frame and subsequentlypooling them in a deep learning framework. We show the effectiveness of ourproposed pooling method on standard benchmarks where it consistently improveson baseline pooling methods, with both RGB and optical flow based Convolutionalnetworks. Further, in combination with complementary video representations, weshow results that are competitive with respect to the state-of-the-art resultson two challenging and publicly available benchmark datasets.
展开▼
